CERN Tape Archive: a distributed, reliable and scalable scheduling system

Proposal for a Web-Based Lecture Archive System for CERN

This document outlines a proposal for the development of an electronic archive system for lectures, tutorials and other slide-based presentations at CERN. The program is based on the implementation of existing software designed to synchronize electronic slide presentations with audio/video recordings through an integrated web interface featuring full indexing and controlled playback. The system...

Scalable Job Scheduling on a Distributed Environment

CScale - A Programming Model for Scalable and Reliable Distributed Applications

Today’s connected world demands applications that are responsive, always available, and can service a large number of users. However, the task of writing such applications is daunting, even for experienced developers. We propose CScale, a programming model that attempts to simplify this task. The objective of CScale is to let programmers specify their application’s core logic declaratively with...

Scaling a Reliable Distributed System

We consider the problem of reliably connecting an arbitrarily large set of computers (nodes) with communication channels. Reliability means here the ability, for any two nodes, to remain connected (i.e., their ability to communicate) with probability at least μ, despite the very fact that every other node or channel has an independent probability λ of failing. A simple solution to the problem c...

TaskMaster: A Scalable, Reliable Queuing Infrastructure for Building Distributed Systems

TaskMaster is a system for managing priority-ordered queues that is designed to scale to 1 billion tasks across 100 thousand queues per node. A reliable queuing system, such as TaskMaster, provides a mechanism for distributing units of inherently serial work (tasks) to workers. Priorities are lexicographically ordered strings that give users more power than FIFO or fixed-range integer prioritie...